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[57] ABSTRACT 

A picture processing apparatus for use in recognizing an 
ambient environment as a picture in accordance with 
the invention, includes a picture input for receiving a 
distorted picture of a wide angle of view, a picture 
processing unit which corrects the distorted picture of 
the wide angle of view from the picture input into an 
undistorted picture of the wide angle of view, and a 
display for displaying the undistorted picture of the 
wide angle of view produced by the picture processing 
unit. 

8 Claims, 10 Drawing Figures 
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BACKGROUND OF THE INVENTION 



to 
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The present invention relates to picture processing 
apparatuses, and more particularly to a picture process- 
ing apparatus which can process a picture of wide angle 
of view at high precision. 

In recent years, moving systems such as a moving 
inspection robot, and intelligent robot etc. have been 
demanded. For moving systems of this type, it is neces- 
sary to function within an environment that is not per- 
fectly known in advance. 

An expedient for recognizing the ambient environ- 
ment of the moving system has heretofore been such 
that the ambient environment is observed with a televi- 
sion camera or the like, and that an operator or a con- 
trolling computer forms a plan for the action of the 
moving system on the basis of a picture obtained by the 
observation. An example of such a system is disclosed in 20 
"Manipulator Vehicles of the Nuclear Emergency Bri- 
gade in the Federal Republic of Germany" contained in 
the Proceeding of 24 th Conference on Remote Systems 
Technology, 1976, pp. 196-218. A serious obstacle to 
information processing for forming a plan of action lies 25 
in the inadequate processing ability of the conventional 
television camera. More specifically, the performance 
of the present-day television camera is greatly inferior 

to that of the human eye. For example, it is known that • , 

. • * r • j irm tHe present invention; 

the human eye can process a picture of wide angle of 30 jr ^ 

view and that it can maintain a high resolution for an 

area worthy of special interest. In order to realize these 

functions with the television camera, a prior art system 

uses a lens of narrow angle of view in compliance with 

a required resolution and changes the direction of the 35 

television camera thereby to scan the whole field of 

view. 

This measure* however, must rotate or turn a televi- 
sion camera of great weight over a large angle. It is 
therefore very difficult to process at high speed a pic 
ture corresponding to the whole field of view. There is 
also a system wherein a large number of television cam- 
eras facing in various directions within the whole field 
of view are positioned for enhancing the picture pro- 
cessing speed. Since, with this system, the. moving sys- 
tem is equipped with the large number of television 
cameras, its weight increases much more, resulting in 
such problems that the cost rises and that the maneuver- 
ability of the moving system degrades. 



environment which does not contain the optical distor- 
tion introduced by the optical system, and display 
means for displaying the undistorted picture of the wide 
angle of view from said arithmetic processing means. 

Other objects, advantages and features of the present 
invention will become more apparent from the follow- 
ing description of embodiments taken with reference to 
the drawings. 



BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a view showing a moving system which is 
furnished with an example of the apparatus of the pres- 
ent invention; 

FIG. 2 is a circuit diagram showing the arrangement 
of a control unit which is used in the apparatus of the 
present invention; 

FIG 3 is an explanatory diagram showing the projec- 
tive conversion operation of an optical system which is 
used in the apparatus of the present invention; 

FIG. 4 is a flow chart showing the processing con- 
tents of the picture processing unit shown in FIG. 2, 
which constitutes the apparatus of the present inven- 
tion; 

FIG. 5 is a diagram showing a picture of a situation 
which is obtained with the apparatus of the present 
invention; 

FIGS. 6 to 8 are diagrams each showing drive means 
for a television camera which is used in the apparatus of 
>n; 

showing a moving system which is 
furnished with another example of the apparatus of the 
present invention; and 

FIG. 10 is a diagram showing the relationship be- 
tween a picture on a monitor television set and the 
resolution thereof, the picture being obtained with the 
example of the apparatus of the present invention 
shown in FIG. 9. 
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SUMMARY OF THE INVENTION 

An object of the present invention is to provide a 
picture processing apparatus which can monitor an 
ambient environment over a wide range. 

Another object of the present invention is to provide 
a picture processing apparatus which can process a 
picture of wide angle of view at high speed. 

A picture processing apparatus for processing a fac- 
simile picture of an ambient environment which is 
scanned through a wide angle optical system to elimi- 
nate optical distortion in the picture that is introduced 
by the optical system in accordance with the invention, 
comprises picture input means for producing a facsimile 
picture of a wide angle of view of the ambient environ- 
ment which contains optical distortion introduced by 
the wide angle optical system, picture processing means 
which corrects the distorted picture of the ambient 
environment into an undistorted picture of the ambient 



PREFERRED EMBODIMENTS OF THE 
INVENTION 

Now, embodiments of the present invention will be 
described with reference to the drawings. 

FIG 1 shows a moving system which is furnished 
with an example of the apparatus of the present inven- 
tion. Referring to the figure, numeral 1 designates the 
moving system which can travel on a stairway, etc. In 
this example, the form of a moving object equipped 
with a crawler 1A is illustrated. Shown at numeral 2 is 
50 a console which is installed in a remote place. This 
console 2 can maneuver the moving system 1 by apply- 
ing a maneuvering signal to the moving system 1 by 
means of a transmitter-receiver 3. The console 2 is also 
provided with a monitor television set 4 which displays 
55 a picture of wide angle of view to be described later. 
The aforementioned moving system is furnished with 
picture input means 5. The picture of wide angle of 
view obtained with the picture input means 5 is pro- 
jected on the monitor television set 4 of the console 2 
60 through the transmitter-receiver 3. As shown in FIG. 1, 
the picture input means 5 is constructed of a convex 
mirror SA and a television camera 5B. Owing to the 
adoption of the convex mirror 5A, the picture of wide 
angle of view can be picked up by the television camera 
65 5B. 

FIG. 2 shows a control circuit in an example of the 
apparatus of the present invention. In this figure, the 
same symbols as in FIG. 1 indicate the same parts. Nu- 
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meral 6 indicates a picture processing unit which elimi- field of view, however, the television camera 5B may be 

nates and compensates the distortion of a picture af- constructed to be rockable relative to the convex mirror 

forded by the convex mirror 5A. This picture process- 5A as shown in FIG. 6. Further, as shown in FIG. 7, the 

ing unit 6 is constructed of picture memory portions 6A, television camera 5B may be constructed to be movable 

6B and a computing circuit 6C. It is possible to install 5 rectilinearly relative to the convex mirror 5A and trans- 

the picture processing unit 6 on the moving system 1 or i atec j by an actuator. It is also possible to tilt or more the 

the console 2. The picture taken by the camera 5B is convex mirror 5A relative to the television camera 5B. 

recorded, in the picture memory portion 6A, while the Further, only the lens portion of the television camera 

picture with the distortion removed is recorded in the 5B may be moved ^ shown in pjQ 8 1{ is also p0S sib>le 

picture memory portion 6B. The computing circuit 6C 10 t0 com bine these measures. Since drive means therefor 

performs a computation for compensating the distortion cafl be readi , Te3A[Z ed, the details shall be omitted from 

of the picture on the basis of a picture position on the thfi description . Moreover, only an area to be noted can 

convex mirror 5A. ^ reccivcd in the whole fic i d of view of the camcra 5B 

^ 1 ! OCC ^Sf ? ° f the f com T P" t,n « c,rcmt 60 by changing the focal length of the lens of the television 

w,H be described with reference to FIGJ l5 J movement of the camera 5B and the 

FIG. 3 is an explanatory diagram showing a projec- A . . . . , . . . . 

tive conversion operation by the convex mirror which adjustment of the focal length m series are executed m 

is used in the apparatus of the present invention. In this accordance with external instructions. The external 

figure, the convex mirror 5A projects an image OR instructions can be given as the operators instructions 

within an infinitely extending plane F, on F A within a 20 by way of example. 

circle of radius T* The projection is done symmetrically It is to be understood that the invention can be prac- 

with respect to an optic axis S. This signifies that, when ticed with a mirror other than the convex mirror 5A, 

the point P of the image OR within the plane F is ex- for example, a mirror of a cone or polygonal pyramid or 

pressed by a polar coordinate system P=(R, 0), the a mirror of a circular or polygonal cylinder, 

projection point P' of the point P on the position ¥a is 25 FIG. 9 shows another example of the apparatus of the 

denoted by (r, 0) in the polar coordinate system. Here, present invention. In this figure, the same symbols as in 

r satisfies the following relation of Equation (1) by way FIG. 1 indicate the same portions. In this embodiment, 

of example: the picture input means 5 is constructed of fisheye 

lenses 5C and television cameras 5B. Also in the case of 

(l) 3Q this embodiment, the distorted picture of a wide angle 
of view can be corrected into the proper picture which 
is displayed on the monitor television set, by the ar- 

✓ R \ 2 rangement of the computing circuitry similar to that in 

1 + [ ~ J the foregoing embodiment shown in FIG. 1. In the case 

35 of this embodiment, since the fisheye lens 5C is used, the 

.m. i_ • j*t« j.- six *t. *• • picture obtained with the corresponding television cam- 
On the basis of Equation (1), the computing circuit y eD . o nfAC ■ . ■* , c ^ r trt f , A 

. . . 4 . J . , , t>i era 5B is more compressed in the area closer to tne 

6C executes processing steps mentioned below. The c+u^gm «p„:.„, tu,„ ^c~i,.f™ 

flow chart of the processing is shown in FIG. 4. I*"P heral ed S e of the field of view Thus the resolution 

(1) The computing circuit reads image data located at of ; he Peripheral part of the picture is degraded. In 
the (r, 8) address of the picture memory portion 6A. 40 order to compensate for this drawback, an area of 
The image data is in the form of a plurality of pixels * h " h a high resolution is required may be scanned 
which comprise the image. continually at the center of the screen However, con- 

(2) It computes the following equation (2) obtained on tmually monitoring the subject area at the center of the 
the basis of the aforementioned equation (1): screen makes it necessary to frequent ly move the televi. 

45 sion camera SB. An expedient for solving this problem 
will be described with reference to FIG. 10. FIG. 10 
shows the relationship between the picture on the moni- 
tor television set as afforded by the fisheye lenses and 
the resolution thereof. T on the picture indicates an 

f_r\ 2 50 equal resolution line, and X, X' indicate subject areas. It 

\J 1 ~ \^r 0 J is now assumed that the area of which a resolution L 0 is 

required has been picked up at the position X on the 

where R, 0 identify the pixels. s ^ e£ ; n * In this ^ the computing circuitry computes 

(3) It writes the image data for each of the pixels of the distance r 0 of the equal resolution line correspond- 
undistorted image at the (R, 0) address of the picture 55 ing to the preset resolution U and the television cam- 
memory portion 6B. cra 5B B moved on tne basis of tne dlstance r o so that the 
As shown in FIG. 5, the picture obtained by the area X may come to the position X' on r 0 . 

above processing is the proper picture WA into which As described above in detail, according to the present 
the picked-up image W including the distortion has invention, a picture of wide angle of view can be pro- 
been corrected and which is displayed on the screen of 60 cessed at high speed so as to provide a proper picture 
the monitor television set 4. The ambient field of view without distortion. As a result, an ambient environment 
of the moving system 1 can be grasped by directly view- over a wide range can be monitored, 
ing the picture WA on the screen of the monitor televi- What is claimed is: 

sion set 4. 1* A picture processing apparatus for processing a 

In the above embodiment, the case has been stated 65 facsimile picture of an ambient environment which is 

where the television camera 5A is held stationary rela- scanned through a wide angle optical system to elimi- 

tive to the convex mirror 5A in the view-field picture nate optical distortion in the picture that is introduced 

input means 5. In order to more enlarge the range of the by the optical system, comprising picture input means 
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for producing a facsimile picture comprised of a plural- which intersects the optical axis and the second side of 

ity of pixels of a wide angle of view of the ambient the triangle being defined by the equation 
environment which contains optical distortion intro- 
duced by the wide angle optical system, said picture 
input means being mounted on a movable support 5 
means which moves within the ambient environment, 

picture processing means which corrects the distorted ^ , 
picture of the ambient environment into an undistorted 
picture of the ambient environment which does not 
contain the optical distortion introduced by the optical 10 

system, said picture processing means having a first wherdn r is ^ iQ r<> &in $ ^ is a fadius of CUfva . 

picture memory means winch stores the pixels of the ture of the CQnvex mirror 

distorted pictoe of the ambient environment a comput- 3 . A icture processing a aratus accord ing to claim 

ing means which produces an undistorted picture of the t „,w«« i™,* ™™ ~~ J(lM * , 

°, . , . * u v • *• j ^ «~ 1, wnerem said picture mput means comprises an optical 

ambient environment by eliminating and compensating 15 • • „ mo ^„ c „ tUi „u #x™ „ „ ^ r .,- .„ : r *t, & 

the distortion of the picture stored in said first picture ? / • 8 i t 
memory means for elich stored pixel of the distorted Th/c " • ° ' * CV1S1 ° n ^ 
picture by computing an address for storage of each 15 f** 0 * 6 m a manner t0 80311 an °P ac . axis of 
stored pixel which will produce the undistorted picture, °P" cal $ ° M ?? WC ? V f ^ P1CtUrC ° f the 
and a second picture memory means which stores the 20 ™de **e 0 f view from said optical viewing means, 
undistortioned picture produced by said computing , 4. A picture processing apparatus according to claim 
means by storing each pixel at the computed address to ?> wherein said optical viewing means is a mirror which 
produce the undistorted picture, display means for dis- 18 convex toward said television camera, 
playing the undistorted picture of the wide angle of 5 * A P ictur e processing apparatus according to claim 
view from said picture processing means; and said pic- 25 3 » wherein said television camera includes means for 
ture processing means and the display means are in- creating relative movement between said optical view- 
stalled at a position which is remote from the movable in S means and the television camera, 
support means, and said picture input means and said 6. A picture processing apparatus according to claim 
picture means are coupled together by communication 3, wherein said optical viewing means is stationary with 
means. 50 respect to said television camera except for a movable 

2. A picture processing system in accordance with lens, 

claim 1 wherein the storage address of each pixel in the 7. A picture processing apparatus according to claim 

second picture memory means is defined in polar coor- 3, wherein said television camera has a variable focus- 

dinates 6 and R where 6 is the angle between the optical ing means. 

axis and a hypotenuse of a right triangle having two 35 8. A picture processing apparatus according to claim 
additional sides with the first side extending from the 3, wherein said optical viewing means comprises a fish- 
center of a convex mirror which is part of the wide eye lens. 

angle optical system to a reflective surface of the mirror * * * * * 
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